A stereo sensor, at a high level, is a sensor that forms a single product, result, or output from inputs simultaneously received from a pair of sensors or detectors. For example, a stereo camera is a pair of cameras that generate a single view of an imaged entity or location from image information received from both cameras. Each camera in a stereo camera has a field of view (FOV), and the fields of view of the two cameras can be combined to give an overall field of view for the stereo camera. In a stereo camera, the fields of view tend to overlap.
The “stereo” nature of a stereo sensor allows for the determination of range information. It can also enable imaging in 3 dimensions, rather than only two dimensions.
Stereo cameras are well known, and have been used in many applications. As examples, stereo cameras have been found to have particular utility in providing three-dimensional (3D) imaging for mapping environments and navigating through them. In such uses, it is not uncommon to use multiple stereo cameras to increase the overall field of view of a system that uses such stereo cameras as an input. For example, U.S. Pat. No. 7,446,766 demonstrates a use of stereo cameras for building evidence grids representing a physical environment and navigating through the environment.
Acquisition of range data can be useful for a number of applications including field measurement, 3-D modeling, navigation and gesture control. There are numerous methods for acquiring range data, including high resolution methods. This concept relates more specifically, but not exclusively, to these high resolution techniques.
Digital imaging technology can be used to capture images of scenes. Comparing images of the scenes taken simultaneously from different vantage points can provide information relating to the range of objects within the scene from the location of the cameras. This technique shall be referred to herein as “stereo ranging.” Other techniques for capturing range information include using patterns of light projected onto the scene, which are then processed by a single imager (e.g., as in the Kinect sensor from Microsoft), or measuring the time of flight of reflections of laser beams scanned across the scene (e.g., the Velodyne HDL-64E or Sick LMS 100 scanners). These alternate methods are employed because they provide higher precision than stereo ranging, while employing simpler data processing techniques. This simpler processing requirement is offset by numerous drawbacks. First, the systems employ active, energy emitting devices, such as LEDs or lasers, which means that they require more energy than systems using multiple imagers. It also means that they cannot be used in applications where active radiation is unwanted or unsafe, such as in military reconnaissance. Also, the range of such systems is limited by the strength of the laser or light source, which makes them impractical for portable devices or in large spaces.
The benefits of using stereo image ranging techniques overcome these drawbacks and provide additional benefits. However, until now, it has been impractical to achieve adequate, robust performance because it is necessary to maintain precise knowledge of the relative position of all of the pixels of one imager and its related optics with the pixels of a second imager and its related optics. Previous attempts at designing and implementing stereo imager pairs have employed techniques to mount individual camera systems, including their electronic interfaces to precision manufactured carriers. Manufacturing the stereo system from individual components is costly, and cannot maintain the relative positions of the imagers under all conditions (see, e.g., Point Grey Research, Inc. U.S. Pat. No. 6,392,688).
FIG. 1 is a perspective view of an example of a prior art stereo camera comprising two independent camera systems mounted onto a metal carrier, which serves as a common support. Two cameras 1 are fixedly mounted to a common support 2 using typical camera mounting techniques, e.g., mounting screws. The individual cameras 1 are not designed with high precision location of the imager and lenses with respect to the mounting points. Furthermore, even if metal carrier 2 is formed of a stiff material, the otherwise physical independence of the cameras, manner of mounting such cameras, materials used to form the cameras, and weight of the cameras are all factors that can result in relative motion between the cameras during usage. As such, the positions of the cameras with respect to each other are liable to move due to thermal expansion, vibration, shock, etc. As a result, the cameras require frequent recalibration to achieve precise stereo ranging—particularly where the apparatus is part of a vehicle or other moving platform. Without such calibration, stereo camera performance can be unacceptably degraded.